System and method for calculating multi-resolution dynamic ambient occlusion

ABSTRACT

A system and method for generating a three-dimensional image is provided. An embodiment of the present invention includes calculating the ambient occlusion at a vertex in multiple, independent stages. Determining the global AO at the vertex may be performed using a first technique. Determining the local AO at the vertex may be performed using a second technique. The total AO can be found as a function of the local AO and global AO.

CROSS REFERENCE TO RELATED APPLICATION

The present application claims the benefit of U.S. provisionalapplication Ser. No. 60/886,387 filed on Jan. 24,2007. The disclosure ofthe provisional application is incorporated herein by reference in itsentirety.

FIELD OF THE INVENTION

The present disclosure relates generally to three-dimensional computergraphics and, more particularly, to a method for facilitatingmulti-resolution dynamic ambient occlusion in real-time videoapplications.

BACKGROUND

Computer-generated, three-dimensional images are now commonly seen inmovies, television shows, video games, and various other videoapplications. Different shading techniques, of varying complexity, arefrequently used to add realism to the generated images and to makedetails more visible. An example of a simple shading technique might beto calculate the effects caused by a single point source of light on animage without taking into consideration factors such as reflection anddiffusion. A more complicated shading technique might be full globalillumination which takes into consideration all direct and indirectlight sources as well as the reflection and diffusion caused by thosesources. Ambient Occlusion (AO) is a shading technique that approximatesfull global illumination by taking into account the attenuation of lightdue to occlusion in order to improve local reflection models. Unlikeother local shading methods, AO determines the illumination at eachpoint as a function of other geometry in the scene.

More complicated shading techniques typically require more calculations,and thus take longer to perform and require more processing resourcesbecause the amount of data used to determine the shading is far greater.Performing these calculations tends to not be a limiting factor formovies or television because those images do not need to be generated inreal time. In applications such as video games, however, images need tobe generated in real time. Therefore, the amount of time allowed andprocessor resources available for generating images are limited byhardware capabilities.

Accordingly, there exists in the art a need to perform shadingtechniques such as ambient occlusion as rapidly as possible so that theycan be incorporated into systems generating real-time images.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a flow chart illustrating an exemplary method according toan embodiment of the present invention.

FIG. 2A shows a vertex that experiences AO from a near object.

FIG. 2B shows a vertex that experiences AO from a distant object.

FIG. 2C shows a vertex that experiences AO from both a near object and adistant object.

FIG. 3A shows an example of how the spheres of a distant object can beused to calculate the global AO experienced by a vertex.

FIG. 3B shows an example of how the spheres of a distant object can beused to calculate the global AO experienced by a vertex.

FIG. 4A shows an example of how triangles can be used to calculate thelocal AO experienced by a vertex.

FIG. 4B shows an example of how triangles can be used to calculate thelocal AO experienced by a vertex.

FIG. 4C shows an example of how triangles can be used to calculate thelocal AO experienced by a vertex.

FIG. 5 illustrates a method that can be used for determining the AOexperienced at a surface element.

FIG. 6 shows a basic system that can be used to implement the methoddescribed in FIG. 1 and/or other embodiments.

DETAILED DESCRIPTION

Embodiments of the present invention include calculating the ambientocclusion experienced by an object in multiple, independent stages. Forexample, a first stage might involve calculating the AO contribution ofnearby, or local, geometry for each vertex of a dynamic model and asecond stage might involve calculating the AO contribution of distant,or global, geometry for each vertex. The AO contributions calculated ineach, independent stage can be merged according to the laws ofprobability in order to find the total AO effect experienced by anobject.

It has been observed that the AO contribution of distant geometry can bewell approximated by the AO contribution of simple geometric primitives,such as sets of spheres. The AO contribution of nearby geometry on theother hand requires more accurate calculations and needs to take theexact underlying geometry into account. Thus, an aspect of the presentinvention includes calculating the AO effects experienced by a vertex inmultiple, independent stages. For example, a first stage might calculatethe AO contribution of nearby, or local, geometry and a second stagemight calculate the AO contribution of distant, or global, geometry.Because AO can be thought of as the probability of occlusion of ambientlight, the AO contributions calculated in each, independent stage can bemerged according to the laws of probability in order to find the totalAO effect experienced by a vertex. By determining the AO at a series ofvertices, a system can generate more realistic and more detailedobjects, which result in higher quality images.

FIG. 1 shows a flow chart illustrating an example method according to anembodiment of the present invention. In this embodiment, the methodbegins when a new frame is generated (block 110). The frame contains oneor more three-dimensional objects that can be approximated by usinggeometric primitives such as spheres and modeled on what are calledmeshes. Meshes can be created from series of interconnected triangles.In response to an event, the sphere and vertex geometry of the image mayneed to be updated (block 120). In a video game, an example of an eventcausing the sphere and vertex geometry of the image to be updated may bea user commanding an object, such as a person or an automobile, to moverelative to other objects in the image.

After the sphere and vertex geometry has been updated, the AOexperienced by the object needs to be calculated. An aspect of thepresent invention includes calculating the total AO in multiple,independent stages. For example, the AO contributed by distant objectsis calculated first (block 130). Then, the AO contributed by the nearobjects are calculated (block 140). Once both have been calculated, thetotal AO can be calculated (block 150). Once the total AO has beencalculated, a new frame including the total AO shading can be generated(block 110), and the process can repeat itself. It will be readilyapparent to one skilled in the art that the order in which the globaland local AO are determined can be altered without the use of inventivefaculty. The local and global AO may also be determined simultaneously.

In a video game system which needs to generate images in real-time, thecalculations needed to generate an image, including the calculations todetermine the total AO, must be performed rapidly.

An embodiment of the present invention involves approximating a modelwith a set of spheres in addition to the triangulated geometry. Afurther embodiment of the present invention includes using a firsttechnique for calculating the contributions of the local AO (block 140),and a second, faster technique for calculating the contributions of theglobal AO (block 130). The global AO component typically varies smoothlyacross a model surface, while the local AO component often changes quiteabruptly. Thus, for each unique vertex in an object, the global AOcomponent may be approximated using the spheres and the local AOcomponent can be calculated using the triangulated geometry.

The determination of whether to use the local technique or the globaltechnique for determining the AO experienced at a vertex by an objectcan be determined either statically by a computer programmer writing thegraphics generating code or may be determined dynamically by variousalgorithms available in the art.

A further embodiment of the present invention involves pruning redundantvertices so that calculations are performed only for unique vertices.For example, when the sphere and vertex geometry is updated (block 120),the system determines for which vertices the AO needs to be determined.Uniqueness can be determined, for example, by vertex position byassuming a smooth model surface.

In another embodiment of the present invention, the AO calculations usedto determine the global AO and local AO may be performed in screen space(i.e., per pixel) and/or in object space (i.e., per vertex). In anexample embodiment of the AO calculations in the algorithm of thepresent invention are performed per pixel with the calculations beingdone on the GPU, instead of, for example, a CPU.

FIGS. 2A, 2B, and 2C show a vertex (v_(i)) 210 that experiences AO 221,222 from both a near object (t_(j)) 230 and a distant object (S_(j))240. The total AO 221, 222 experienced by the vertex (v_(i)) 210 will bea function of both the local AO 221 and the global AO 222.

FIGS. 3A and 3B show an example embodiment involving use of spheres 320to approximate a distant object can be used to calculate the global AOcontribution of the total AO experienced by a vertex (v_(i)) 310. Asphere 320 with center position p_(s) and radius r_(s) will occlude avertex v_(i) 310 with position p_(v) and normal n_(v) 330 if it is, atleast partially, above v_(i)'s tangent plane. The AO is a function ofthe angle α of the spherical cap that is defined by p_(v) and thetangents of the sphere that go through p_(v). The normalizedcosine-weighted solid angle of a spherical cap with angle α is sin² α.

FIG. 3A illustrates a first case where the sphere 320 is fully visible.The AO can be calculated as follows:

v = p_(s) − p_(v) ${\sin^{2}\alpha} = \frac{r_{s}^{2}}{v^{T} \cdot v}$${\cos\;\beta} = {\frac{v^{T}}{v} \cdot n_{v}}$ AO = 1 − sin²α cos  β

FIG. 3B illustrates a second case, where the sphere 320 is clipped bythe tangent plane. The AO can be approximated as follows:

v = p_(s) − p_(v) ${\sin^{2}\alpha} = \frac{r_{s}^{2}}{v^{T} \cdot v}$${\cos\;\beta} = {\frac{v^{T}}{v} \cdot n_{v}}$${\cos\;\beta^{\prime}} = \frac{{\cos\;\beta} + {\sin\;\alpha}}{2}$AO = 1 − sin²α cos  β^(′)

FIGS. 4A, 4B, and 4C show example embodiments involving the use oftriangles to calculate the local AO contribution of the total AOexperienced by a vertex (v_(i)) 410. To calculate the local AO of avertex v_(i) 410 with position p_(v) and normal n_(v) 450, two types oflocal triangles may be considered because a vertex (v_(i)) 410 caneither be part of a triangle or not. FIG. 4A shows an example of atriangle that does not include vertex v_(i). A triangle 420 with vertexpositions p₀, p₁, p₂ (i.e., it does not include p_(v)) will occludev_(i) 410 if it is, at least partially, above v_(i)'s tangent plane andfront-facing with respect to v_(i) 410. The calculations can be done byfirst projecting the triangle 420 vertices onto the unit sphere 430centered at p_(v). The solid angle of the resulting spherical triangle440 can either be calculated exactly using the Euler-Eriksson formula orapproximately by treating it as a planar triangle. The angle α of aspherical cap with the same solid angle can then be obtained. Bytreating the spherical triangle 440 as a spherical cap that is centeredat the spherical triangle's centroid, the calculations can then proceedas follows:

$d_{0} = \frac{p_{0} - p_{v}}{{p_{0} - p_{v}}}$$d_{1} = \frac{p_{1} - p_{v}}{{p_{1} - p_{v}}}$$d_{2} = \frac{p_{2} - p_{v}}{{p_{2} - p_{v}}}$$d_{c} = \frac{d_{0} + d_{1} + d_{2}}{{d_{0} + d_{1} + d_{2}}}$$\omega = {2\;\arctan\frac{{d_{0} \cdot \left( {d_{1} \times d_{2}} \right)}}{1 + {d_{0} \cdot d_{1}} + {d_{1} \cdot d_{2}} + {d_{2} \cdot d_{0}}}}$$\omega \approx {\frac{1}{2}{{\left( {d_{2} - d_{0}} \right) \times \left( {d_{1} - d_{0}} \right)}}}$${\cos\;\alpha} = {\left. {1 - \frac{\omega}{2\pi}}\Rightarrow{\sin^{2}\alpha} \right. = {1 - \left( {1 - \frac{\omega}{2\pi}} \right)^{2}}}$cos  β = d_(c)^(T) ⋅ n_(v) AO = 1 − sin²α cos  β

A triangle with vertex positions p₀, p₁, p_(v) will occlude v_(i) if atleast one vertex is above v_(i)'s tangent plane. In the followingcalculations it is assumed (without loss of generality) that p₀ is notbelow p₁ with respect to v_(i).

FIG. 4B shows an embodiment in which both vertices are above v_(i)'stangent plane. The resulting spherical quadrilateral can be evaluated byevaluating the two spherical triangles Δd₀d₁d₂ and Δd₁d₂d₃:

$d_{0} = \frac{p_{0} - p_{v}}{{p_{0} - p_{v}}}$$d_{1} = \frac{p_{1} - p_{v}}{{p_{1} - p_{v}}}$$d_{2} = \frac{p_{0} - p_{v} - {\left( {\left( {p_{0} - p_{v}} \right)^{T} \cdot n_{v}} \right) \cdot n_{v}}}{{p_{0} - p_{v} - {\left( {\left( {p_{0} - p_{v}} \right)^{T} \cdot n_{v}} \right) \cdot n_{v}}}}$$d_{3} = \frac{p_{1} - p_{v} - {\left( {\left( {p_{1} - p_{v}} \right)^{T} \cdot n_{v}} \right) \cdot n_{v}}}{{p_{1} - p_{v} - {\left( {\left( {p_{1} - p_{v}} \right)^{T} \cdot n_{v}} \right) \cdot n_{v}}}}$

FIG. 4C shows an embodiment where p₀ is above v_(i)'s tangent plane andp₁ is below it, resulting in the spherical triangle Δd₀d₂d₄:

$d_{0} = \frac{p_{0} - p_{v}}{{p_{0} - p_{v}}}$$d_{2} = \frac{p_{0} - p_{v} - {\left( {\left( {p_{0} - p_{v}} \right)^{T} \cdot n_{v}} \right) \cdot n_{v}}}{{p_{0} - p_{v} - {\left( {\left( {p_{0} - p_{v}} \right)^{T} \cdot n_{v}} \right) \cdot n_{v}}}}$$d_{4} = \frac{p_{0} - p_{v} - {\left( \frac{\left( {p_{0} - p_{v}} \right)^{T} \cdot n_{v}}{\left( {p_{1} - p_{0}} \right)^{T} \cdot n_{v}} \right) \cdot \left( {p_{1} - p_{0}} \right)}}{{p_{0} - p_{v} - {\left( \frac{\left( {p_{0} - p_{v}} \right)^{T} \cdot n_{v}}{\left( {p_{1} - p_{0}} \right)^{T} \cdot n_{v}} \right) \cdot \left( {p_{1} - p_{0}} \right)}}}$

The calculated AO represents the probability of no occlusion. Therefore,the total AO can be obtained by multiplying the global AO and local AOcomponents.

The complexity of the method can be determined byO(n_(s)*n)+O(n_(t)*n)=O(n), where n_(s) and n_(t) are the average numberof contributing spheres and triangles respectively per vertex and n isthe number of unique vertices. The sphere set approximation can enablethe global AO calculations to be independent of the resolution of thedistant geometry and, as a result, the complexity is O(n) rather thanI(n²). Additional speed-up can be achieved by playing back a “fullmotion range” animation in a pre-processing stage and figuring out theset of spheres and local triangles that could potentially contribute tothe global and local AO of each unique vertex, making it possible toonly evaluate a sub-set of all spheres and local triangles at run-time.The global AO changes smoothly over time, meaning it can be updated lessfrequently than the local AO, thus, providing another speed-upmechanism.

In an embodiment, the model triangles can be used in a pre-processingpass to approximate the model surface by a set of surface elements. Eachunique model vertex can have one corresponding surface element, whereuniqueness can be determined by vertex position alone (essentiallyassuming a “smooth” model surface). A surface element can be atwo-dimensional (2D) object (e.g., a disc) that is defined by its centerposition, normal direction and area. In this implementation, the centerposition and the normal direction of each surface element can be takendirectly from the associated vertex data and can be continuously updatedat run-time. The surface element area is calculated by summing the areasof all the adjacent triangles of the associated vertex and dividing bythree (meaning that each triangle contributes as much area to each ofits three vertices).

The real-time calculations can then find the amount of AO at the center,r_(i), of each surface element. Each surface element's position andnormal can define a plane, conceptually the tangent plane of the modelsurface at that point. Other surface elements that are located (at leastpartially) above the tangent plane will hence contribute to the ambientocclusion as long as they are back facing with respect to r_(i).

For illustrative purposes, FIG. 5 provides an example method that can beused for determining the AO at r_(i). For each surface element r_(i), atangent plane using r_(i)'s position, p_(i), and normal, n_(i) can bedefined. For each other, back-facing, surface element e_(j) the AOcontribution can be calculated and added.

There are many techniques available for calculating the ambientocclusion contribution of a surface element. According to M. Bunnell,Dynamic Ambient Occlusion and Indirect Lighting, GPU Gems 2, NVIDIACorp. (2005), repeating the above calculations multiple times whilefeeding back the approximate ambient occlusion values calculated thusfar can improve the accuracy of the solution. In the implementationprovided herein, the result of the previous frame's calculations can bere-used to increase accuracy, which can speed up processing.

While the complexity of the basic method is O(n²), where n is the numberof surface elements, the performance can be improved to O(n log n) byhierarchically merging distant surface elements to form larger “virtual”surface elements. Even though the needed merging data can be created ina pre-processing stage it still introduces additional complexity to therun-time algorithm. Instead of implementing this, nearby surfaceelements can be grouped and each group can be enclosed by a boundingsphere. This allows quick discard of entire groups of surface elementsin the calculations if the associated bounding spheres were found to notcontribute to the ambient occlusion, thereby providing additional, muchneeded, processing speed-up.

FIG. 6 shows a basic system that can be used to implement the variousembodiments described in FIG. 1 and/or other embodiments. The systemincludes a central processing unit (CPU) 610 connected to a data storagemodule 620. The data storage module 620 may be a combination of harddrives, RAM, ROM, or any other type of machine-readable storage deviceknown in the art, and may be configured to contain instructions operableto cause the CPU 610 to carry out the method. The CPU 610 may be anysort of data processing element available such as those commonly foundin video game consoles or personal computers. In an embodiment, the CPU610 further is configured to receive signals from an input device 630such as a keyboard, mouse, remote control, video game controller, or anysort of input device known in the art. The input device 630 may be usedby a user of the system to cause the CPU 610 to generate images in realtime. The generated images may be output to a display 640 such as acomputer monitor or television through a graphics processing unit (GPU)650 such as a video card. In an alternative embodiment, the method maybe carried out either completely or partially within the GPU 650. Asecondary data storage module 660 may be accessible to the GPU 650 andmay store some or all of the instructions operable to cause the GPU 650to carry out the method.

The foregoing description of embodiments is provided to enable a personskilled in the art to make and use the present invention. Variousmodifications to these embodiments will be readily apparent to thoseskilled in the art, and the generic principles and specific examplesdefined herein may be applied to other embodiments without the use ofinventive faculty. For example, some or all of the features of thedifferent embodiments discussed above may be deleted from the embodimentand/or may be combined with features of alternate embodiments.Therefore, the present invention is not intended to be limited to theembodiments described herein but is to be accorded the widest scopedefined only by the claims below and equivalents thereof.

1. A computer program product tangibly embodied in a machine-readablestorage device, the product comprising instructions operable to cause adata processing apparatus to: receive a signal to produce an image, theimage to include a first object; determine the location of a vertex ofthe first object; determine a global ambient occlusion (AO) at thevertex of the first object using a first technique; determine a localambient occlusion (AO) at the vertex of the first object using a secondtechnique; determine a total ambient occlusion (AO) at the vertex of thefirst object as a function of the local AO and global AO; and, producethe image, the shading at the vertex of the first object to be dependenton the total AO, wherein the image includes other objects formed ontriangle-based models comprising a plurality of triangles andapproximated by sphere-based models comprising a plurality of spheres,wherein the local AO is determined as a function of the geometry of aset of triangles in the triangle-based model.
 2. A computer programproduct tangibly embodied in a machine-readable storage device, theproduct comprising instructions operable to cause a data processingapparatus to: receive a signal to produce an image, the image to includea first object; determine the location of a vertex of the first object;determine a global ambient occlusion (AO) at the vertex of the firstobject using a first technique; determine a local ambient occlusion (AO)at the vertex of the first object using a second technique; determine atotal ambient occlusion (AO) at the vertex of the first object as afunction of the local AO and global AO; and, produce the image, theshading at the vertex of the first object to be dependent on the totalAO, wherein the image includes other objects formed on triangle-basedmodels comprising a plurality of triangles and approximated bysphere-based models comprising a plurality of spheres, wherein theglobal AO is determined as a function of the geometry of a set ofspheres in the sphere-based model.
 3. A computer program producttangibly embodied in a machine-readable storage device, the productcomprising instructions operable to cause a data processing apparatusto: receive a signal to produce an image, the image to contain a firstobject; determine the location of a vertex of the first object;determine a global ambient occlusion (AO) at the vertex of the firstobject using a first technique; determine a local ambient occlusion (AO)at the vertex of the first object using a second technique; determine atotal ambient occlusion (AO) at the vertex of the first object as afunction of the local AO and global AO; produce the image, the shadingat the vertex of the first object to be dependent on the total AO;determine if a second vertex is a redundant vertex; and, upondetermining that a second vertex is a redundant vertex, not determininga total AO for the second vertex.
 4. A computer program product tangiblyembodied in a machine-readable storage device, the product comprisinginstructions operable to cause a data processing apparatus to: receive asignal to produce an image, the image to contain a first object;determine the location of a vertex of the first object; determine aglobal ambient occlusion (AO) at the vertex of the first object using afirst technique; determine a local ambient occlusion (AO) at the vertexof the first object using a second technique; determine a total ambientocclusion (AO) at the vertex of the first object as a function of thelocal AO and global AO; produce the image, the shading at the vertex ofthe first object to be dependent on the total AO; and, determine a setof spheres of the plurality of spheres that do not contribute to the AOat the vertex.
 5. A computer program product tangibly embodied in amachine-readable storage device, the product comprising instructionsoperable to cause a data processing apparatus to: receive a signal toproduce an image, the image to contain a first object; determine thelocation of a vertex of the first object; determine a global ambientocclusion (AO) at the vertex of the first object using a firsttechnique; determine a local ambient occlusion (AO) at the vertex of thefirst object using a second technique; determine a total ambientocclusion (AO) at the vertex of the first object as a function of thelocal AO and global AO; produce the image, the shading at the vertex ofthe first object to be dependent on the total AO; and determine a set oftriangles of the plurality of triangles that do not contribute to the AOat the vertex.
 6. The computer program product of claim 1, wherein thedata processing apparatus is a video game console.
 7. The computerprogram product of claim 2, wherein the data processing apparatus is avideo game console.
 8. The computer program product of claim 3, whereinthe data processing apparatus is a video game console.
 9. The computerprogram product of claim 4, wherein the data processing apparatus is avideo game console.
 10. The computer program product of claim 5, whereinthe data processing apparatus is a video game console.